Analog VLSI Stochastic Perturbative Learning Architectures∗

We present analog VLSI neuromorphic architectures for a general class of learning tasks, which include supervised learning, reinforcement learning, and temporal difference learning. The presented architectures are parallel, cellular, sparse in global interconnects, distributed in representation, and robust to noise and mismatches in the implementation. They use a parallel stochastic perturbation technique to estimate the effect of weight changes on network outputs, rather than calculating derivatives based on a model of the network. This “model-free” technique avoids errors due to mismatches in the physical implementation of the network, and more generally allows to train networks of which the exact characteristics and structure are not known. With additional mechanisms of reinforcement learning, networks of fairly general structure are trained effectively from an arbitrarily supplied reward signal. No prior assumptions are required on the structure of the network nor on the specifics of the desired network response.